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Abstract — The problem of minimizing queueing delay of op- 
portunistic access of multiple continuous time Markov channels 
is considered. A new access policy based on myopic sensing 
and adaptive transmission (MS-AT) is proposed. Under the 
framework of risk sensitive constrained Markov decision process 
with effective bandwidth as a measure of queueing delay, it 
is shown that MS-AT achieves simultaneously throughput and 
delay optimality. It is shown further that both the effective 
bandwidth and the throughput of MS-AT are two-segment 
piece-wise linear functions of the collision constraint (maximum 
allowable conditional collision probability) with the effective 
bandwidth and throughput coinciding in the regime of tight 
collision constraints. Analytical and simulations comparisons 
with the myopic sensing and memoryless transmission (MS-MT) 
policy which is throughput optimal but delay suboptimal in the 
regime of tight collision constraints. 

Index terms — Delay optimal medium access, effective bandwidth, 
opportunistic access, and constrained risk sensitive Markov 
decision process. 

I. Introduction 

WE consider in this paper delay optimal policies for a 
secondary user in a multichannel hierarchical overlay 
cognitive network (T), where a cognitive user can sense and 
transmit on one of the TV channels assigned to the primary 
users. A secondary user should only transmit on a channel 
where the primary user is not transmitting, and its trans- 
missions are subject to interference constraints imposed by 
primary incumbents. 

In such hierarchical cognitive networks, transmission op- 
portunities for a secondary user depend on random traffic 
patterns of the primary users, which makes it necessary that 
the secondary user has a queue that holds the arrival packets. 
Thus packets of a secondary transmitter are subjects to random 
delays. For applications with delay constraints, there is a need 
to find a sensing and transmission policies that minimize 
queueing delays caused by random transmission opportunities 
and transmission failures due to collisions with the primary 
users. 

Effective bandwidth (or effective capacity in the terminol- 
ogy used in j2)-||4l) is an indirect way to measure the delay 
performance of a policy for applications either with bursty 
random arrivals at or random departures from the transmitter 
queue; the latter may be caused by channel fading as in 13, (3| 
or random transmission opportunities considered in this paper. 
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To characterize the delay due to random arrivals or departures, 
Kelly argued in that "constraints on the probability that 
buffer space or delay exceeds a certain threshold is more 
important than constraints on mean values." The idea is that 
a provider offering services should guarantee the subscriber 
a measure of "effective bandwidth" that takes into account 
randomness either in the arrival or the departure processes. 
This viewpoint, implicitly espoused earlier in [6|], led to the 
class of approaches based on the theory of large deviations. 
See iTTTl — im~ll and references therein. 

In the context of opportunistic secondary transmissions, the 
notion of effective bandwidth is defined as follows. Given the 
buffer size t > 1 of the secondary transmitter and e, the 
effective bandwidth is the maximum arrival rate a(e, b) of the 
secondary user traffic such that, despite the randomness in 
transmission opportunities and collisions with primary users, 
the probability of the queue length exceeding b is capped 
below e. For applications with constant rate of arrivals, the 
queue size of a secondary user is proportional to packet delays. 
Therefore the effective bandwidth can be interpreted as the 
sustainable arrival rate with delay constraint. 

On the surface, transmission delays caused random opportu- 
nities resembles that caused by fading, which has been studied 
in El-E). There are, however, some important differences. 
What we are interested in are delay optimal policies that 
choose which channel to sense and the way to transmit; the 
problem of finding an optimal policy was not the objective of 
EJ-JU. In addition, the collision constraints imposed in this 
paper do not have a correspondence in the problems studied 
earlier. 

A. Summary of Results and Contribution 

We consider TV identical and independent primary channels, 
modeled by on-off continuous time Markov processes, with 
"on" indicating the channel is being used by the primary 
user and "off a transmission opportunity for the secondary 
user. The secondary user adopts a slotted sensing-before- 
transmission protocol which is defined by a sensing policy 
that determines which channel to sense and a transmission 
policy that specifies the probability of transmission. 

If the secondary user transmits on a particular channel, a 
collision with the primary user is possible because, as the 
licensed incumbent, the primary user transmits whenever it 
has a packet. We assume that the secondary user abides by 
the interference constraints imposed by the primary users. 
Here the interference to a primary user is measured by the 
collision probability between the primary and secondary users 
conditioned on the event that the primary user transmits. In 
this paper, the set of admissible policies includes those whose 
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interference to primary channels is uniformly bounded above 
by 7. 

The main results of this paper are as follows. First, we 
propose a simple policy referred to as MS -AT (myopic sensing 
and adaptive transmission). The sensing policy is the myopic 
policy originally proposed by Zhao, Krishnamachari, and Liu 
|[T2| for slotted primary user systems. The key idea of MS- 
AT is an adaptive transmission policy that adjust adaptively 
its transmission probabilities based on feedbacks of past 
transmissions (ACK/NAK). In particular, given the maximum 
acceptable collision constraint 7, the adaptive transmission 
policy sets a target rate r(7) of successful transmissions and 
adjust its transmission probabilities based on the estimated rate 
of successful transmissions. We note that the proposed MS-AT 
policy can be interpreted as a debt-based policy whose prin- 
ciple is used to establish feasible throughput optimality in the 
deadline scheduling problem lfT3l - lfT31 . though the deadline 
scheduling problem and the debt-based policy considered in 
|[T3l - |fT5l are very different from those in this paper. 

Second, we establish that MS-AT simultaneously achieves 
delay and throughput optimality under any collision constraint. 
In establishing the optimality of MS-AT, the main idea is 
to formulate the problem of maximizing effective bandwidth 
subject to collision constraints as one of risk sensitive Markov 
decision process which adopts an exponential objective func- 
tion (instead of the usual linear objective function in the risk 
neutral Markov decision process). In particular, in the regime 
of loose collision constraints, the optimality of MS-AT is 
shown by the technique of Hernandez and Marcus who estab- 
lished the multiplicative dynamic programming representation 
of the unconstrained risk sensitive Markov decision process in 

m. 

Third, we provide simple characterizations of the throughput 
and effective bandwidth as functions of the collision constraint 
parameter 7. It is shown that both the effective bandwidth and 
the throughput of MS-AT are two-segment piece-wise linear 
functions of 7, and the effective bandwidth and throughput co- 
incide in the regime of tight collision constraints as illustrated 
in Fig. Q] 

The result that the effective bandwidth matches to the 
throughput in the regime of tight collision constraints seems 
surprising since the effective bandwidth is typically strictly 
smaller than the throughput [17|. A commonly accepted intu- 
ition is that the randomness in the input process requires more 
bandwidth, and randomness in the output process decreases 
the sustainable rate traffic if the delay is required to be small. 
This intuition is consistent with our result when the collision 
constraints are loose as it can be observed in Section [TV] 
that under loose collision constraints the optimal effective 
bandwidth is strictly smaller than the optimal throughput. 
However, when the collision constraint is tight, the delay 
optimal policy offers effective bandwidth equal to the optimal 
throughput (see Theorem [TJ. The key insight, as presented 
in Section [III] is that the secondary user can make its packet 
departure process favorable by an adaptive transmission policy. 
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Fig. 1. Illustration of throughput and effective bandwidth of MS-AT as 
piece-wise linear functions of collision constraints. 

B. Related Work 

Substantial amount of work exists for opportunistic access 
with throughput as the performance measure. In (T8j, Zhao 
et. al. consider the case when primary users follow a slotted 
Markov transmission structure under the per slot interfer- 
ence constraint. The myopic sensing policy is shown to be 
throughput optimal for identical, positively correlated Marko- 
vian channels lfT2l . fl9l . The continuous time Markovian 
occupancy model has also been treated using the throughput 
measure (see, e.g., l20l - l22l ). Adopting a periodic channel 
sensing policy, the optimal transmission policy is obtained 
in the framework of constrained Markov decision process 
under the average interference constraint (22), which further 
leads to the throughput optimality of the periodic sensing with 
memoryless transmission (PS-MT) policy when the collision 
constraints are tighQ l23l , ll24l . The multiuser maximum 
throughput region of opportunistic access is studied in ESI 
in which the throughput optimal policies employ memoryless 
transmission policy. 

The use of effective bandwidth to analyze delay for oppor- 
tunistic spectrum access was first considered in |26| where the 
authors obtain in closed form the effective bandwidth of the 
myopic sensing policy with slotted primary transmission, with- 
out establishing the optimality of myopic sensing. In (27 1 both 
throughput and effective bandwidth are considered for multi- 
user cognitive access. The effective bandwidth optimality of 
myopic sensing for two identical Markov channels under loose 
collision constraints is obtained in 11271 . whereas our result in 
this paper establishes the optimality for a general number of 
homogeneous channels under loose collision constraint as well 
as new optimal policy for general collision constraint. In 11281 
the authors propose an ALOHA based policy for multiuser 
opportunistic access and analyze its delay performance via a 
fluid model. 

The optimization of effective bandwidth under tight colli- 
sion constraint can be formulated as constrained risk sensitive 

'the tight collision constraint in 1231 . 1241 is different from the tight 
collision regime for MS-MT 
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Markov decision. The only existing results to our knowledge 
are |29l , 11301 . where |29l considers the set of Markov policies 
whereas li30l treats the general constrained Markov decision 
process with the objective function and the constraints given 
by general utility functions and establishes the existence of an 
optimal policy in the set of general history dependent policies 
without any structural property or computation procedure of 
the optimal policy. In this paper, we exploit the special form 
of the utility function and the associated linear constraints, 
which leads to a structured optimal policy. 

II. Network Model 

The network model and the assumptions are described as 
follows. The hierarchical overlay cognitive network consists 
of N parallel primary channels indexed by 1, . . . ,N and a 
secondary user. Each primary user transmits on its dedicated 
channel, the occupancy of which is assumed to be independent, 
identically distributed continuous time Markov process. The 
state space of the channel is {l(busy), O(idle)} and the holding 
times are exponentially distributed with mean /i -1 and A -1 
for busy and idle states, respectively. The secondary user 
opportunistically accesses the primary channels in a slotted 
sensing-before-transmission manner with slot length T. In 
each time slot the secondary user senses one out of the 
N channels (sensing is assumed to be perfect) and decides 
whether or not to transmit in the sensed channel. Successful 
transmission of c bits is achieved in slot t if the secondary 
user transmits in a channel that is idle throughout slot t. In 
particular, if the secondary user senses a channel to be busy in 
slot t, no transmission will be attempted in this slot since no 
successful transmission but collision will be achieved. We aim 
to maximize the effective bandwidth subject to the collision 
constraints imposed by the primary users. 

A. Effective Bandwidth 

Effective bandwidth characterizes the quantity of service 
available to the secondary user via the opportunistic channel 
with QoS constraint. To give the definition of the effective 
bandwidth we consider the queueing process of the secondary 
user. 

Assume that the incoming traffic of the secondary user is a 
constant arrival process with intensity of a bits per slot, and 
the bits are stored in a buffer of size 6^1 before being 
transmitted. For a fixed opportunistic access policy tt, denote 
by the queue length at the end of slot t. Then (Q^)t>o 
satisfies the following recursion 



Q1 = niax{Q 



t-i 



-i?f,0}, t>l, Q£ = 0. 



where (R^) t >o is the output process of the opportunistic 
channels (K£ = c if successful transmission is achieved in 
slot t, and RJ = otherwise). Under regularity conditions, 
Glynn and Whitt show in JH) that, if the policy tt is such that 
the queue is stable, then converges in distribution to a 
steady state distribution with the decay rate 



lim -logPr(Q^ >x) = -ff*{a), 

X^QO X 



where 9 n (a) is the unique positive solution of the equation 

a6 + VK-6) = 0, (2) 

and ^(0) is the Gartner-Ellis limit 

1 " 
lim -logE^exp^i^)]. 

n— too n 

k=l 

The decay rate 9* (a) in Eq. ([TJ depends on the secondary 
user's traffic arrival rate a and the output process (Rt)t>o\ 
the latter depends on the primary channel statistics and the 
opportunistic access policy. With the decay rate in Eq. (fTJ the 
packet drop (buffer overflow) probability can be approximated 
by 

Pr[Q^ > 6] « exp(-0»&), (3) 

for large buffer size b. 

With the constraint that the buffer overflow probability is 
capped by e and adopting the large buffer approximation in 
Eq. ©, the effective bandwidth a v (e, b) can be defined as 



o 7r (e,6) = max{a : cxp(-b9" (a)) < e}. 



(4) 



It can then be shown (see l26l ) that the following effective 
bandwidth formula holds. 



i*(e,b) = lim 



(5) 



The above expression shows a key connection between 
effective bandwidth and throughput. Applying the Jensen's 
inequality, we have a v (e,b) < lim n _ ! . 00 ^-E^ Y^t=i R-t ■ m 
general, the effective bandwidth is strictly smaller than the 
throughput, except in certain special scenarios (see Section 
fTTTb . Asymptotically as the buffer size approaches infinity 
(b oo) or the QoS requirement is relaxed (e — ^ 1), the 
effective bandwidth does converge to the throughput. 

B. Collision Constraints 

The transmissions of the secondary user are subject to colli- 
sion constraints imposed by the primary users. The collision is 
measured in a long term average manner. Specifically, for each 
primary user i the scaled infinite horizon average collision with 
primary user i must be bounded by 7; the scale infinite horizon 
average collision for primary user i is 



C71V 



lim 



1 — v(0) cxp(— AT) n->oo n 



1 n 

n — ' 



(i) 



where Xi,t ls me indicator of collision with primary user i in 
slot t, and the scale uses the reciprocal of 1 — v(0) exp(— AT) 
(u(0) = + n) is the stationary distribution of idle state), 
i.e., the steady state probability of primary user i transmitting 
in a certain slot. Given the collision parameter 7, the set of ad- 
missible policies 11(7) is given by the set of policies that meet 
the interference constraints, i.e., 11(7) = {tt : C„ i < 7,Vz}. 
We aim to characterize the maximum effective bandwidth for 
the admissible policy set 11(7). 
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Fig. 2. Illustration of MS-MT policy. Filled(Open) circle: secondary user 
decides to(not to) transmit. 



C. Opportunistic Access Policy 

The opportunistic access policy consists of two components: 
the channel sensing policy which selects a channel to sense 
in each slot based on the past history, and the transmission 
policy which specifies the transmission probability upon idle 
sensing results. (Upon busy sensing results the secondary user 
will keep silent.) 

We first describe the myopic sensing policy (MS). In lfl2l . 
lfl9l . myopic sensing is shown to be throughput optimal 
for independent, identically distributed, positively correlated, 
discrete time Markov channels with a simple round-robin 
structure. Specifically, the secondary user first fixes an ordered 
list of the N channels. To start, the secondary user senses the 
first channel in the list and keeps sensing it until the first 
busy sensing result. Then the secondary user switches to the 
next channel in the list and keeps sensing this channel. The 
secondary user cycles through the channel list in this stay- 
when-idle, switch-when-busy manner. 

When it comes to the transmission policy, the transmission 
probability is determined, in general, based on the entire 
history. A simple (but in general suboptimal) transmission 
policy is the memoryless transmission (MT) policies where 
the transmission decision depends only on the current sensing 
outcome with constant probability of transmission upon idle 
sensing results. We consider in this paper a new class of 
transmission policy referred to as adaptive transmission (AT) 
policies. As discussed in detail in Section[III] the transmission 
strategies at time slot t depends on the history of transmissions 
in the past. 

In this paper, we consider two types of opportunistic ac- 
cess policies: MS-MT — the myopic sensing and memoryless 
transmission (25), l27l (with constant transmission probability 
p us chosen to satisfy the interference constraints) and a new 
policy referred to as MS -AT — myopic sensing and adaptive 
transmission. Figs. [2] illustrates the sample path of the MS- 
MT policy and the MS-AT policy is described in Section IITT1 

III. Optimality of MS-AT 

First define 9 = log(e)/& to be the effective bandwidth 
parameter for buffer size b and packet drop probability e. 
Rewrite the effective bandwidth formula (© with 6 



a*(e,b) = lim 
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A. Adaptive Transmission Policy 

According to the effective bandwidth formula © a simple 
upper bound can be easily obtained, i.e., the optimal through- 
put under collision parameter 7. To achieve this upper bound, 
we first describe the mechanism of adaptive transmission pol- 
icy (AT). We employ the acknowledgement of the secondary 
user to aid the transmission decision. Specifically, after sensing 
a channel to be idle in slot t the secondary user counts the 
total number of acknowledgements received up to slot t — 1, 
denoted by At and transmits with probability 1 if At < rt. 
Otherwise if At > rt, the secondary user stays silent. Here r 
is the parameter of AT policy and controls the collision caused 
by the secondary user. 

The adaptive transmission policy can be also thought of as 
a debt based transmission policy. The target of the secondary 
user is set to be transmitting rt packets within the first t slots, 
thus if A t < rt, the secondary user is in debt and thus needs 
to transmits with probability 1, and otherwise, if A t > rt, the 
secondary user has a reasonable balance and stays silent. 

The optimality of the MS-AT policy with respect to effective 
bandwidth and the optimal effective bandwidth as a function 
of the collision parameter 7 is provided in Theorem Q] 

Theorem 1: For any collision parameter 7, there exists r 
such that the MS-AT policy with parameter r is feasible and 
optimal for the constrained effective bandwidth optimization 
with collision parameter 7. The parameter r of the optimal 
MS-AT policy is given by 

jV7Cxp(-Ar)(l- U (0)cxp(-AT)) 
1 — cxp(— XI ) 

where v(0) — p/(X + fi) is the stationary distribution of idle 
state. The optimal effective bandwidth is given by 



. r jV 7 exp(-Ar)(l-i;(0)exp(-Ar)) * x 
e,d — mini 



Ms(oo) 



(0) 



where 



1 - exp(-AT) 

log(l-exp(-AT)(l-exp(c0))), 

1 n 
MS (°°)(0)= lim -logE^^rexp^V i?" s(c 

n— s-oo n — ' 
fc=l 



}• 



(7) 
(8) 



and vp 1 ^ 00 ' (9)/ 6 is the optimal effective bandwidth when the 
collision parameter 7 is loose (the minimum in Eq. (|7]i is 
assumed by the second term). 

Also the MS-AT policy with parameter r is optimal with 
respect to the throughput, with optimal throughput 

JV7exp(-AT)(l-u(0)exp(-AT)) 



TH* 



min{- 



1 



1E MS(00) J2 R 
t=l 



1 - exp(-AT) 
}• 



Ms(oo) ^ 
t 



Theorem[T]provides the solution to the constrained effective 
bandwidth optimization problem by establishing the optimality 
of MS-AT for all collision parameter 7. To prove Theorem Q] 
we need to first prove a list of lemmas. In the later presentation 
we assume c = 1 without loss of generality. 
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B. MS-AT with r = oo 



define 



We first establish the effective bandwidth optimality of MS- 
AT policy with r = oo under loose collision parameter 7. We 
denote the MS-AT policy with r = 00 by MS-AT(oo). By 
the structure of MS-AT, the secondary user always transmits 
upon idle sensing results under MS-AT(oo) policy. Denote by 
^ms(oo) m( ji ca tor of the secondary user having an idle 
sensing result in slot t, following MS -AT (00) policy. 

Call a collision parameter 7 loose if the MS -AT (00) policy 
is admissible with respect to 7. We are interested in proving 
that the MS-AT(oo) policy has better effective bandwidth 
than any other admissible opportunistic access policy. To this 
end, we relax the collision constraint and turn to show that 
the MS-AT(oo) policy has better effective bandwidth than 
any other opportunistic access policy, whether admissible or 
not. This relaxation leads to an unconstrained risk sensitive 
Markov decision process, the optimality of which is provided 
in Lemma Q] 

Lemma 1: Under loose collision constraint, MS-AT(oo) is 
effective bandwidth optimal over all admissible policies in 

n( 7 ). 

Proof: In view of Eq. <(5j and the fact that the effective 
bandwidth parameter 8 = log(e)/6 < 0, to show the effective 
bandwidth optimality we just need to show that MS-AT(oo) 
minimizes M* n (6) = E T exp(6< YlLi R l) over a11 n e n (7) 
for all n > 1. We then make connection between Mjj n {6) and 

M x.n( e ) = E^cxp^^tU^D' where X t is the 'indicator 
of the secondary user having an idle sensing result in slot t 
under opportunistic access policy it. Specifically, we have the 
following inequality 

MIJ9) = E[E[e 9 ^"=1 R * I X*, . . . , X*]] 

n 

= E[IlE[e^ \Xn\ 

t=l 

n 

> E[[l - (sup^)e~ AT (l - e ce )]£"=i A ""] (9) 
t=i 

= M£ >B (log(l - (suptf)e- XT (l - e ce )M0) 
t=i 

where ^ is the transmission probability used by policy 7r in 
slot t. 

Under loose collision constraint it holds for opportunistic 
access policy it that ^ < 1, On the other hand, the MS- 
AT(oo) policy always transmit with probability 1. Therefore 
equality holds in Eq. (0 for MS-AT(oo). Thus we just need 
to show that for an arbitrary opportunistic access policy tt, 



M x (9)>M^\e) 



(11) 



for all 6 < 0. 

We prove Eq. ( fTTT i with the following multiplicative dy- 
namic programming recursion |fl6l for the finite horizon 
problem with Ee e 5D&=i Xk as the objective function. First 



Vk-M = min Ee ex ^ ] 

l<a<N 

V t (u) = min E[e 9X "^V t+1 (T(LJ,a))} 

l<a<N 

for t = 1, ...,K — 1, where a is the dummy variable 
denoting the current action at time t of choosing to sense 
the ath channel, and Vt(-) is the value function, defined as 
the minimum expected future objective function that can be 
achieved starting from t when the information state is u>, i.e., 
Vt(u>) = inf,,- E,r[exp(0 5ZJ^_ t .Xfc) | u>], where the informa- 
tion state u) is the length N vector with the ith component 
being the conditional probability that the ith primary channel 
will be sensed idle in the next slot (see |[T9l for the definition 
and interpretation of the channel information state vector u>). 
Expanding the dynamic programming recursion based on the 
sensing outcome, we have 

V K (u>) = min Ee ex » M = min \u) a e e + (1 - uj a )\ 

^ ' l<a<N l<a<N l V J> 

V t (u) = min E[e ex ^V t+ i(T(u>,a))) 

l<a<N 

= min {u) a e e V t+ i{T(oJ,a \ 0)) 

l<a<N 

+(l-u a )V t+ i(T(u,a) I 1)} 

We can proceed from here and show that picking the largest 
component in the channel information state vector u) to sense 
solves the multiplicative dynamic programming equation, and 
thus prove the effective bandwidth optimality of MS-MT, 
following the line of lemmas in fl9l . One key lemma in |[T9l 
(Lemma 3) needs to be modified to the multiplicative version 

e e V t+ i(uJi, . . .,w T ,_2,Pn,Poi) > V t +i(poi,wi, ■ ■ ■,w„_2,pn). 

We omit the details of the proof due to limited space. ■ 
Note that under loose collision constraints as 7 increases the 
optimal effective bandwidth stays constant; the optimal value 
is 



MS(oo) 



(6) *" s(oo) (6i) 



EB*(e,b) = 
where 6 = log(e)/6, and 

6 = log(l - exp(-AT)(l - exp(ce))), 
and the last equality follows from Eq. (TTOb . 



(12) 



C. Structural Properties of MS-AT 

We analyze the structure of the adaptive transmission policy 
(AT) coupled with myopic channel sensing policy. Specif- 
ically, there will be consecutive time slots in which the 
secondary user is in good balance (therefore stays silent) and 
is in debt (therefore keeps transmitting upon idle sensing 
results), respectively. We denote the consecutive intervals 

by Ii, ...,/„,.. . and Bi, . . . , B ni We also denote by 

i\, . . . , i n , . . . and bi, . . . ,b n , . . . the first time slot in the 
intervals 7i, . . . , I n , . . . and B\, . . . , B„, . . .. Note that in the 
first slot it always holds that A\ = < r, which indicates 
that the interval B\ {B n ) comes before interval I\ (I n )- 
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Lemma 2: As r increases, the effective bandwidth associ- 
ated with MS-AT with parameter r is non-decreasing. 

Proof: Prove this lemma with a sample path argument. 
Specifically, for arbitrary n < r 2 and any horizon n we show 
that along any sample path of the channel state realization the 
MS-AT policy with t 2 accumulates more successful transmis- 
sions than the MS-AT policy with n, i.e., 

n n 

t=i t=i 

where the lower case r indicates the quantities are conditioned 
on a sample path realization. Eq. $13[ will follow if one can 
show for any integer fc that the MS-AT policy with t 2 achieves 
the fcth successful transmission no later than the MS-AT policy 
with Ti for the specific channel state realization. 

Assume otherwise, i.e, for some integer k the fcth successful 
transmission occurs earlier under the MS-AT policy with t\. 
Without loss of generality assume that the k is the smallest 
among the integers that satisfy this condition. Thus a contra- 
diction can be easily drawn since in the time slot of the fcth 
successful transmission under the MS-AT policy with ti, the 
MS-AT policy with t 2 has more debt than the MS-AT policy 
with ti, and thus will decide to transmit, which is guaranteed 
to be successful since the MS-AT policy with n has the fcth 
successful transmission in that very time slot. Therefore the 
contradiction proves Eq. (13[ , and the claim of the lemma 
follows. ■ 

Lemma 3: For all fc > 1 it holds that 

A ik > i k r,A bk < b k r, (14) 

and 

A ih _ t < (i k - l)r, A bk ^ > (b k - l)r. (15) 

Proof: We prove the inequalities using the alternating 
nature of the sequence of intervals we define and the structure 
of the MS-AT policy. 

By the structure of the adaptive transmission policy, in time 
slot t if the MS-AT policy attempts to transmit (i.e., t £ B k 
for some fc), then A t < rt. Similarly, if the MS-AT policy 
stays silent (i.e., t e I k for some fc), then A t > rt. 

For all fc > 1, by definition i k is the first time slot in the fcth 
idle period, therefore Ai k > i k r. For all fc > 1, by definition 
i k — 1 is the last time slot before the fcth idle period, therefore 
ik — 1 lies in a certain busy interval, which indicates A; t _i < 
(ik — 1)t. The other two inequalities can be shown similarly, 
except that there is one special case for k = 1 (61 — 1 = 
does not lie in any idle or busy interval). To accommodate this 
special case we define by convention Aq = 0. ■ 

Lemma 4: For any r < ^^ s(oo) (0) /Q, 

1) the lengths of the intervals I k are bounded by - almost 
surely, 

2) the lengths of the intervals Bk have finite expectation. 

Proof: In time slot 1, the secondary user is always 
transmitting since A\ = < r. Therefore the fcth idle interval 
Ik is ahead of the fcth busy interval B k . Thus the idle interval 



Ik starts from slot ik (inclusion) and ends in slot bk+i — 1 
(inclusion). Therefore the length of the idle interval Ik is 

\h\ = h+i - ik- 

According to the structure of MS-AT policy and Lemma [3] 
it holds that 

A lk < A ik ^ x + c < (i k - 1)t + c, 

A bk+1 > A 6fe+1 _i > fe+ i - 1)t, 

and because during the idle interval Ik nothing is transmitted, 
it follows that A bk+1 = Ai k . Therefore 

(b k +i - 1)t < (i k - l)r + c, 

which indicates 

141 = Ofc+i - i k < - = -■ 

T T 

In other words, idle periods cannot be longer than 1/r 
since every 1/r slots there will be r • - = 1 packet due 
for transmission based on the debt based interpretation of the 
adaptive transmission policy. 

The busy interval B k begins from slot bk (inclusion) and 
ends in slot ik — 1 (inclusion). According to Lemma[3]it holds 
that A bk < rbk and 

A bk > A bk -i > rb k - t. (16) 

This can be interpreted as follows. In time slot bk the MS- 
AT policy has accumulated A bk <E [rbk — T,rbk) successful 
transmissions, which is still short compared with the target 
rb k . 

By definition slot ik is the first slot after slot bk in which 
the MS-AT policy has no debt. Again according to Lemma [3] 
it holds that A; fc > rik and 

A lk < Ai k -i + c<T(i k -l) + c, (17) 

i.e., the balance in slot ik is less than c — r. 

Now consider the event {\Bk\ = n}. It holds that 

{\B k \=n} = {ik = n + b k } 

c {A n+bk < r(n + b k - 1) + c} (18) 
C {A n+bk - A bk <rn + c} (19) 

= { A " +bfc ~ At <r + -}, (20) 
n n 

where Eq. ( fTBl and Eq. ( fT9l ) make use of Eq. ( fT6b and Eq. 

(1171 , respectively. 

By definition of busy interval the secondary user is always 

transmitting upon idle sensing results in the interval [bk, n— 1 + 

b k ]. Therefore as n tends to infinity the following convergence 

holds exponentially fast since the joint state which combines 

the channel state vector and the current channel index the 

secondary user is at evolves as a finite state recurrent Markov 

chain (see ||25) Section IV.B), 

1 1 " 

lim -E Ms(T ^(A n+bk -A bk ) = lim -E MS (°°)yi?" s(oo) 

n— too ji n—too fi ' J 

> ^ {co) (6)/9, 
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where the middle term is the optimal throughput of the MS-AT 
policy with r = oo, /.e., the secondary user always transmits 
upon idle sensing results. 

Since \&^p°°^(6>) / > r, for n large enough r + 
c/n will lie to the left of *™ (oo) (0)/0, and thus to the 



left of i^^oo iE Ms (°°) x;r=i i? r (oo) with gap at ieast 



Therefore the set in Eq. d20t corresponds to a deviation of at 



least lim™ iE MS (°°) £" =1 i?! 



MS(oo) 



- * 



Ms(oo) 



(0)/0 and has 



exponentially decaying probability in n, and so does the set 
{\Bk\ = n}, which indicates that the lengths of the intervals 
Bk have finite expectation. ■ 
Lemma 5: For any t < $™ (oo) (0) /0, we have for the MS- 
AT policy with r = oo (MS(oo) policy) 

7i r Ms(oo) —Q rn 

supM R ^ n '(8)e 



< oo, 



(21) 



where M™£°\0) = E MS (°°) exp(0£; i = i R7 {oo) )- 

Proof: Following Eq. ( TTOb one can easily obtain that 



lim -logM*^ (0) = #* s(oo) (0). 



(22) 



Therefore for any r < ^^ OD \d)/6, there exists e > such 
that *^ s(oo) (0)/0 - e - t > 0. By the limit in Eq. {22j, we 



have that there exists A such that for all n > N 



1 



-f-logM^(0)>*7^(0)/0-e 



Ms(oo) , 



which implies 

M^(0)<e 
and further for all n > N 



ie(*^ s(oo) (s)/e- 



(23) 



(24) 



jVe(4" s(oc) (9)/9-e 



(25) 



Therefore we conclude 



max {Af" s j oo) (0)e _f ' T4 }} < oo, 

Ki<N "> % 



thus proving the lemma. ■ 
Lemma 6: For any t < the effective band- 

width of the MS-AT policy with parameter r is no less than 
r. 

Proof: By the definition of effective bandwidth we have 
the effective bandwidth of the secondary user is given by 



a 



ms(t) 



(e, b) = lim 



l 0g E MS M exp(0£ t " =1 i? 



ms(t)^ 



D0 



(26) 



for buffer size b ^> 1 and = log(e)/6. 

Define T(n) to be the slot index that corresponds to the last 
slot before n (inclusion) that is the start of an interval, i.e., 

T(n) — max{i < n : i — ik or i = bk for some fc}. (27) 

Then the analysis deal with two possibilities, T{n) = ik for 
some k and T(n) = bk for some k. 



1) T(n) = ik for some k. Then we have 

= Ee e(AT( " ) ~ rT( " )+E *= T <") R "-r(n-T(n))) 

< Ee e(E^ = T ( „)-Rfc-^(«- T («))) 

< Ee -r|/ fc |e 

< Ee- e 

where we used the inequalities A; fc > rife and |7fc| < 
1/r. Therefore we have 

lim -logEe eE ?=i Rt - Tn) < 0, 

which leads to 

lim 4r lo g E e ei: "= lflt > 7". 

n->oo 710 

2) T(n) = 6fc for some k. Then we have 

Eg^Efci Si"™) 

= Ee e(AT <"'~ rT(ll)+ ^= r <"> «*- T ("- T ("))) 

< e- f,T supM i? . fe (0)e- (,rfc 

k 

where we used the inequality Ab k — rbk > —t and 
Lemma 
Therefore we have 

lim -logEe^"^*-™ 5 <0, 

n— too Ji 

which also leads to 

_1 lim -5- log Ee e ^" =1 ^* > t. 

Therefore we have proved that the effective bandwidth of 
the MS-AT policy is at least as large as the parameter r. ■ 

Lemma 7: The throughput of the MS-AT policy with pa- 
rameter r is at most r. 

Proof: The throughput of the secondary user is given by 

Ih-n^^ ±E MS( -^ J2t=i R T (T) ■ A S ain s P nt the analysis into 
two possibilities, T(n) = ik for some k and T(n) = bk for 
some k. 

1) T{n) = ik for some k. Then we have 

n 

E(]T i? t - rn) 

n 

= E(A T{n) - rT(n) + £ i? fe - r(n - T(n))) 

fc=T(n) 

< E(c-t), (28) 

where Eq. d28l l is due to the fact that from slot T{n) to 
slot rt the secondary user stays silent. Thus it holds that 



1 " 

lim -E( V R t - rn) < 0, 



1=1 



g 



which leads to 

1 ™ 
lim -E > R t < t. 

n— J-oo fl — * 
t=l 

2) T(n) = for some k. Then we have 

n 

t=l 

n 

= E(A T{n) - TT(n) + Rk-r{n-T{n))) 

k=T(n) 

n 

< E E c 

fc=T(n) 

< cE|S fc |. 

Due to the fact that the expected length of a busy period 
is finite, we have in parallel with the previous case 



1 " 
lim -E(> R t - rn) < 

n.— ^no 71 — J 



0. 



which leads to 



lim 



t=i 



Rt < r, 



Therefore we have proved that the throughput of the MS -AT 
policy is at most as large as the parameter r. ■ 

Corollary 1: For any r < ty^°°\o~)/6, the effective band- 
width and throughput of MS-AT with parameter r is both r. 

Proof: Application of Jensen's inequality implies that for 
the same opportunistic access policy ir the effective bandwidth 
is no greater than the throughput linin^oo Ylt=i ^t- 
However, we have already established that the throughput of 
MS-AT is no greater than the parameter r and the effective 
bandwidth is no less than the parameter r. Therefore com- 
bining the two facts, we conclude that the throughput and the 
effective bandwidth are both equal to the parameter r. ■ 

Lemma 8: For any collision parameter 7, the following 
choice of parameter r 

jV 7 exp(-Ar)(l-?;(0)e X p(-AT)) 

T = 1 T~^F\ (29) 

1 — cxp(— \1 ) 

is feasible. 

Proof: The MS-AT policy only transmits upon idle sens- 
ing results. Therefore for any horizon n the ratio between 
the expected number of total successful transmissions and the 
expected number of total collisions is ^xp^-ArP 1 With MS- 
AT policy the total number of successful transmissions the 
secondary user achieves during the first t slots satisfies 



< Tt 



(30) 



Taking the expectation of Eq. fl30b implies that the expected 
number of total successful transmissions is upper bounded by 
rt + c and thus the expected number of total collisions is 
bounded by ( rt + c X^y° ? ^j~ A ' r )) _ Then we divide this upper 
bound expression by t, take the limit t — > 00 and take into 



account the scaling coefficient 1— v(0) exp(— AT) (see Section 
III-BI ). Now we obtain that the sum of the collision C MS ( T ) A < 
Nj. 

Note that the channel sensing policy and the transmission 
policy is symmetric with respect to the N channels. Therefore 
the lemma is proved. ■ 

Then we are ready to prove Theorem Q] 

Proof: Note that the parameter r(7) given by Eq. d29b is 
no less than the best throughput for any opportunistic access 
policy under collision limit 7 (see 11251 ). 

We also know from Corollary Q] that for any r < 
%M°°)0yg ^ effective bandwidth and throughput of MS- 
AT with parameter t is both r. Therefore for any r < 
^x {oo} {6)/9 the effective bandwidth of MS-AT with parame- 
ter t matches with the optimal throughput upper bound, thus 
optimal with respect to effective bandwidth. 

Combining this fact with Lemma [2] and the fact that MS- 
AT(oo) has optimal effective bandwidth ^>^°°\§) / 6 for 
loose collision parameters, we conclude that for any collision 
parameter 7 the parameter r given by 

_ A r 7 cxp(-AT)(1 - v(0) exp(-AT)) 
1 - exp(-AT) 

achieves the optimal effective bandwidth 

iV 7 cxp(-AT)(l -v(0)exp(-AT)) ^ s(oo) (0) 



EB* = min{ 



1 - exp(-AT) 9 

The throughput optimality of the MS-AT policy can be 
proved similarly. ■ 

IV. Simulation Results 

We plot the throughput and effective bandwidth versus the 
collision parameter 7 for MS-AT. In comparison, we also plot 
the throughput and effective bandwidth versus the collision 
parameter 7 for MS-MT, the policy with myopic sensing but 
memoryless transmission. In the simulation the number of 
primary channel is set to be N = 2. The slot length in the 
simulation is taken to be T = 0.25ms. The channel parameters 
we use are fj, = l/2ms _1 and A = l/3ms _1 . The effective 
bandwidth parameter 9 = log(e)/6 is taken to be -0.08. 

Fig.[3]depicts the throughput and effective bandwidth versus 
the collision parameter 7 for the MS-AT and MS-MT policies. 
The plot shows that the effective bandwidth and throughput 
matches up to a certain point for MS-AT policy, and then 
levels up at optimal effective bandwidth for loose collision 
parameter, validating the optimality result for MS-AT policy. 
On the other hand, the throughput and effective bandwidth for 
the MS-MT policy deviates in the entire 7-axis, showing that 
the MS-MT policy is indeed suboptimal when the collision 
constraint is tight. On the other hand, when the collision 
constraint is loose, the two policies both give the optimal 
effective bandwidth. The optimal effective bandwidth deviates 
from the throughput in the loose collision regime, as predicted 
in the presentation. Also note that in Fig. [3] the optimal 
throughput as well as the effective bandwidth is first linearly 
increasing in 7 and then levels up. 
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(a) MS-AT (b) MS-MT 

Fig. 3. Throughput and Effective bandwidth of MS-AT and MS-MT. 




(a) MS-AT (b) MS-MT 

Fig. 4. Empirical distribution of MS-AT and MS-MT. 



The empirical distribution of A t is plotted in Fig. 0] It can 
be observed that under MS-AT policy, the distribution of A t 
is more condensed near the target rt, whereas under MS-MT 
policy, the distribution of A t is more far apart from rt. 

V. Conclusions 

Multichannel opportunistic access of homogeneous contin- 
uous time Markov channels is considered. The objective is 
to design an opportunistic access policy that maximizes the 
effective bandwidth under the interference constraints. This 
paper shows that a myopic sensing plus adaptive transmission 
policy is optimal with respect to both effective bandwidth 
and throughput under all levels of collision constraint. The 
optimality result may find applications in cognitive radio net- 
works for spectrum overlay with QoS requirement. Although 
we focus on single user scenario in this paper, the technique 
can be extended to multiuser scenario and similar results can 
be established for the multiuser effective bandwidth region. 
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